Error Accrual and Mitigation During Robotic Process

ABSTRACT

A robotic system includes end-effector(s) that combine a plurality of objects in a production process. The system includes sensor(s) that obtain measurement(s) relating to a combination of a first object and one or more other objects during the production process. The system includes a control system communicatively coupled to the sensor(s). The control system stores specifications relating to the combination of the plurality of objects. The control system receives the measurement(s) from the sensor(s), determines a difference based on the measurement(s) and the specifications, determines adjustment(s) to the production process based on the determined difference, and sends, for the end-effector(s), instruction(s) based on the specifications and the one or more adjustment(s). The end-effector(s) combine a second object with the first object and the one or more objects based on the specifications and the one or more adjustment(s).

BACKGROUND

A robot may include one or more end-effectors that allow the robot tomanipulate objects and one or more sensors that guide the manipulationof the objects. For example, a robot can use the end-effector(s) andsensor(s) to combine a plurality of objects into a desired physicalstructure.

SUMMARY

To produce a physical structure from a plurality of physical objects, arobotic system may be guided by specifications that are generated from amodel for the physical structure. The actual production process,however, might introduce variations/errors that may affect the finalphysical structure produced. According to aspects of the presentdisclosure, however, the robotic system can monitor each step of theproduction process and dynamically determine how to mitigate the effectof any variations/errors that may be introduced at each step.Specifically, the robotic system can employ sensors to take measurementsafter each step in the production process. Using these measurements, acontrol system of the robotic system can identify any variations/errorsand evaluate whether the production process will produce the desiredphysical structure. The specifications derived from the model providethe robotic system with the necessary flexibility to customize theproduction process so that the desired physical structure is produced.The specifications define a flexible design space where differentparameters in the production process can be modified to accommodatephysical reality. Although the adjustments to the production process maychange aspects of the resulting physical structure, the robotic systemoperates so that the production process preserves the intent of thedesigner as set forth in the model.

According to an example embodiment, a robotic system includes a bodyincluding one or more end-effectors configured to combine a plurality ofobjects in a production process. The system includes one or more sensorsconfigured to obtain one or more measurements relating to thecombination of the plurality of objects during the production process.The system includes a control system including one or more processorsand one or more data storage devices. The control system iscommunicatively coupled to the one or more sensors. The one or more datastorage devices store specifications relating to the combination of theplurality of objects. The control system is configured to: receive, fromthe one or more sensors, one or more first measurements relating to acombination of a first object and one or more other objects during theproduction process; determine a difference based on the one or morefirst measurements and the specifications; determine one or moreadjustments to the production process based on the determineddifference; and send, for the one or more end-effectors, one or moreinstructions based on the specifications and the one or more adjustmentsto the production process. The one or more end-effectors, in response tothe one or more instructions, combines a second object with the firstobject and the one or more other objects based on the specifications andthe one or more adjustments to the production process.

According to another example embodiment, a method for a robotic systemincludes combining, with one or more end-effectors, a first object andone or more other objects during a production process. The methodincludes obtaining, with one or more sensors, one or more measurementsrelating to combining the the first object and the one or more otherobjects during the production process. The method includes storing, onone or more data storage devices, specifications for a combination of aplurality of objects. The method includes determining, with a controlsystem including one or more processors, a difference based on the oneor more measurements and the specifications. The method includesdetermining, with the control system, one or more adjustments to theproduction process based on the determined difference. The methodincludes combining, with the one or more end-effectors, a second objectwith the first object and the one or more other objects based on thespecifications and the one or more adjustments to the productionprocess.

According to yet another embodiment, a method for a robotic systemincludes storing, on one or more data storage devices, a model forcombining a plurality of objects. The method includes combining a firstset of the objects. The method includes obtaining, with one or moresensors, one or more measurements relating to the combination of thefirst set of the objects. The method includes determining, with acontrol system implemented with one or more processors, one or moredifferences based on the model and the one or more measurements. Themethod includes determining, with the control system, one or moreadjustments based on the one or more differences. The method includescombining, with one or more end effectors, a second set of the physicalobjects with the first set of objects based on the model and the one ormore determined adjustments.

These as well as other aspects, advantages, and alternatives will becomeapparent to those of ordinary skill in the art by reading the followingdetailed description with reference where appropriate to theaccompanying drawings. Further, it should be understood that thedescription provided in this summary section and elsewhere in thisdocument is intended to illustrate the claimed subject matter by way ofexample and not by way of limitation.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates an example configuration of a robotic systemaccording to aspects of the present disclosure.

FIG. 2 illustrates an example robot that produces a physical structureat a production site, according to aspects of the present disclosure.

FIG. 3A illustrates an example process for generating specificationsfrom a model and producing a physical structure from the specifications,according to aspects of the present disclosure.

FIG. 3B illustrates further aspects of the example process of FIG. 3A,according to aspects of the present disclosure.

FIG. 4A illustrates an example model for installing physical tiles on awall, according to aspects of the present disclosure.

FIG. 4B illustrates an example process for generating specificationsfrom the model of FIG. 4A and producing a tile installation from thespecifications, according to aspects of the present disclosure.

FIG. 5A illustrates example specifications, including constraints, forinstalling physical tiles on a wall, according to aspects of the presentdisclosure.

FIG. 5B illustrates a step during an example production processemploying the example specifications of FIG. 5A, according to aspects ofthe present disclosure.

FIG. 5C illustrates another step during the example production processof FIG. 5B, according to aspects of the present disclosure.

FIG. 5D illustrates yet another step during the example productionprocess of FIG. 5B, according to aspects of the present disclosure.

FIG. 5E illustrates a further step during the example production processof FIG. 5B, according to aspects of the present disclosure.

FIG. 6 illustrates aspects of another example production process for atable, according to aspects of the present disclosure.

DETAILED DESCRIPTION

The following detailed description describes various features andfunctions of the disclosed systems and methods with reference to theaccompanying figures. The illustrative system and method embodimentsdescribed herein are not meant to be limiting. It will be readilyunderstood that certain aspects of the disclosed systems and methods canbe arranged and combined in a wide variety of different configurations,all of which are contemplated herein.

I. OVERVIEW

A robot may include one or more end-effectors that allow the robot tomanipulate or otherwise engage physical objects and one or more sensorsthat guide the manipulation of the physical objects. A robot can use theend-effector(s) and sensor(s) to combine a plurality of physical objectsinto a desired physical structure. According to one example, the desiredphysical structure may be an airplane wing and the physical objects arethe components, such as spars, ribs, and sheet metal, that are combinedto produce the airplane wing.

To produce this physical structure, the robot may be guided byspecifications that are generated from a model for the physicalstructure. The model may provide a designer's conception of how thephysical structure will be configured and how it will perform whenactually produced by the robot. To develop the model, the designer maydefine a plurality of model objects, which may provide the designer'sconception of what physical objects are combined to produce the physicalstructure. The designer designs each model object to contribute anappropriate set of properties that helps the physical structure providethe desired configuration and performance.

The designer also defines relationships between the model objects. Theserelationships determine how the model objects are combined for thephysical structure. For instance, these relationships may determinepositions/orientations of the model objects relative to other modelobjects. In addition, these relationships may determine how the modelobjects are directly or indirectly coupled to each other. Furthermore,these relationships may determine what manufacturing, assembly, or otherproduction techniques to employ to combine the model objects.

The designer may further define aspects of the model according to amodel production site. The model production site models externalstructures and conditions at the production site where the physicalstructure will be produced. The designer may also determinerelationships between the the model objects and aspects of the modelproduction site.

In the example above, the designer may develop a model that provides anengineer's conception of how spars, ribs, sheet metal, and othercomponents can be combined to produce the airplane wing. The designermodels spars, ribs, sheet metal, and other components to have thedesired properties for the airplane wing. In addition, the designermodels the relationships between the spars, ribs, sheet metal, and othercomponents, indicating for instance how they are positioned/orientedrelative to each other, how they are coupled to each other, and how theyare assembled together according to different production techniques.Furthermore, the designer may model aspects of the factory floor thatmay affect the production of the airplane wing, including for instancetemperature and humidity that may affect the components and theproduction techniques.

Ideally, the robot produces a physical structure that closely resemblesthe model as initially conceived by the designer. However, the actualproduction process might introduce variations/errors that may affect thefinal physical structure.

When developing the model, the designer may attempt to make theproduction process for the physical structure more robust by providingtolerances for acceptable variations introduced by the productionprocess. For instance, the model may include tolerances for smallvariations in the position or orientation of each physical object. Evenwhen the robot installs the physical objects with variations within thetolerances, however, such variations may accumulate to create anunanticipated effect on the final physical structure. Furthermore, therobot may install one or more of the physical objects with errorsoutside of the tolerances. In general, the robot may encounterproduction issues that cannot be sufficiently addressed by thetolerances.

Advantageously, the robot can monitor each step of the productionprocess. For instance, the robot can take various measurements ofvariations/errors within the three-dimensional space of the actualproduction site. The robot can then determine the effect of thevariations/errors on the final physical structure. With the feedbackprovided by such measurements, the robot can dynamically determine howto mitigate the effect of any variations/errors that may be introducedat each step. In some cases, the robot may decide to abort theproduction process. In other cases, the robot can dynamically addressthe effect of variations/errors by determining adjustments for theproduction process. By allowing the robot to respond to thevariations/errors on its own, the production process can continuewithout interruption and/or intervention by operators or the like.

In the example above, the robot may take three-dimensional measurementsof the airplane wing's size and shape as the production processprogresses. From these measurements, the robot can identify differences,i.e., variations/errors, between the actual airplane wing and the model.The robot can assess these differences to decide whether the productionprocess can still produce the desired airplane wing. If necessary, therobot can adjust the production process to account for the differencesand produce the desired airplane wing. If adjustments cannot overcomethe differences, the robot can abort the production process.

Although the adjustments to the production process may change aspects ofthe resulting physical structure, the robot operates so that theproduction process preserves the intent of the designer as set forth inthe model. In other words, the robot can customize the productionprocess to address issues in the production process while stillproducing a physical structure generally consistent with the model.

In general, parameters of the model have modifiable values, settings,etc. As such, the specifications provide a parameter space that allowsfor adjustments. In other words, the adjustments select values,settings, etc., for one or more of the parameters of the model inresponse to the variations/errors. The selection of these values,settings, etc., can optimize the parameter space for the model.

When developing the model, the designer may define one or moreconstraints that establish the scope of allowable adjustments to theproduction process. The specifications communicated to the robot alsoinclude information on the constraints. In particular, the constraintsestablish the parameter space for the adjustments. The constraints mayinclude features that are most significant to the designer and indicatethe designer's intent behind the model. The robot can evaluate theconstraints to determine how to customize the production process for thephysical structure.

After evaluating a variation/error resulting from the installation ofone of the physical objects, the robot might determine that one or moreconstraints cannot be satisfied even if the production process ismodified. In such cases, the robot can abort the production process. Inother words, the robot can make a “go/no go” decision after theinstallation of each physical object.

When developing the model, the designer may also define respectiveweightings for the constraints. With the weightings, the specificationscan further guide customization of the production process. The robot canevaluate the weightings to determine a preferred approach that yields aphysical structure most consistent with the designer's intent. Forinstance, the constraints may be weighted to indicate the relativeimportance that the designer places on each constraint. The weightingsmay indicate that some constraints must be satisfied. Meanwhile, theweightings may indicate that other constraints are preferences which maybe modified (or even ignored) in order to produce the desired physicalstructure. Furthermore, the weightings may indicate the order in whichmodifiable constraints should be modified, i.e., which preferences havehigher priority.

In the example above, the designer of the airplane wing may defineconstraints that require the airplane wing to have a certain aerodynamicprofile that falls within a practical range of profiles. The designermay also define constraints based on safety regulations. If necessary,the robot can then make adjustments to the production process to accountfor the accrual of variations/errors during the production process aslong as the final airplane wing is produced according to the constraintsand their respective weightings.

II. EXAMPLE ROBOTIC SYSTEM

FIG. 1 illustrates an example configuration of a robotic system 100 thatmay be used in connection with the embodiments described herein. Therobotic system 100 may be configured to operate autonomously,semi-autonomously, and/or using directions provided by user(s). Therobotic system 100 may be implemented in various forms, such as a bipedrobot, quadruped robot, a robotic arm, or some other arrangement.Furthermore, the robotic system 100 may also be referred to as a robot,robotic device, or mobile robot, among other designations. Generally,the robotic system 100 provides a device that has a computing abilityand can interact with its surroundings with an actuation capabilityand/or with ability to emit/generate physical phenomena such as lightand/or sound, among others.

As shown in FIG. 1, the robotic system 100 may include processor(s) 102,data storage 104, and controller(s) 108, which together may be part of acontrol system 118. The robotic system 100 may also include sensor(s)112, power source(s) 114, mechanical components 110, and electricalcomponents 116. The robotic system 100 is shown for illustrativepurposes, and may include more or fewer components. The variouscomponents of the robotic system 100 may be connected in any manner,including wired or wireless connections. Further, in some examples,components of the robotic system 100 may be distributed among multiplephysical entities rather than a single physical entity. Other exampleembodiments of the robotic system 100 may exist as well.

Processor(s) 102 may operate as one or more general-purpose hardwareprocessors or special purpose hardware processors (e.g., digital signalprocessors, application specific integrated circuits, etc.). Theprocessor(s) 102 may be configured to execute computer-readable programinstructions 106, and manipulate data 107, both of which are stored inthe data storage 104. The processor(s) 102 may also directly orindirectly interact with other components of the robotic system 100,such as sensor(s) 112, power source(s) 114, mechanical components 110,and/or electrical components 116.

The data storage 104 may be one or more types of hardware memory. Forexample, the data storage 104 may include or take the form of one ormore computer-readable storage media that can be read or accessed by theprocessor(s) 102. The one or more computer-readable storage media caninclude volatile and/or non-volatile storage components, such asoptical, magnetic, organic, or another type of memory or storage, whichcan be integrated in whole or in part with the processor(s) 102. In someembodiments, the data storage 104 can be a single physical device. Inother embodiments, the data storage 104 can be implemented using two ormore physical devices, which may communicate with one another via wiredor wireless communication. As noted previously, the data storage 104 mayinclude the computer-readable program instructions 106 and the data 107.The data 107 may be any type of data, such as configuration data, sensordata, and/or diagnostic data, among other possibilities.

The controller 108 may include one or more electrical circuits, units ofdigital logic, computer chips, and/or microprocessors that areconfigured to (perhaps among other tasks) interface between anycombination of the mechanical components 110, the sensor(s) 112, thepower source(s) 114, the electrical components 116, the control system118, and/or a user of the robotic system 100. In some embodiments, thecontroller 108 may be a purpose-built embedded device for performingspecific operations with one or more subsystems of the robotic device100

The control system 118 may monitor and physically change the operatingconditions of the robotic system 100. In doing so, the control system118 may serve as a link between portions of the robotic system 100, suchas between the mechanical components 110 and/or the electricalcomponents 116. In some instances, the control system 118 may serve asan interface between the robotic system 100 and another computingdevice.

During operation, the control system 118 may communicate with othersystems of the robotic system 100 via wired or wireless connections, andmay further be configured to communicate with one or more users of therobotic system 100. As one possible illustration, the control system 118may receive an input (e.g., from a user or from another robot)indicating an instruction to perform a particular action.

A wired connection in the robotic system 100 may include, for example, aparallel bus or a serial bus such as a Universal Serial Bus (USB).Meanwhile, a wireless connection in the robotic system 100 may include,for example, Bluetooth, IEEE 802.11 (such as IEEE 802.11-2007, IEEE802.11n-2009, or any other IEEE 802.11 revision), cellular (such as GSM,GPRS, CDMA, UMTS, EV-DO, WiMAX, HSPDA, or LTE), or Zigbee, among otherpossibilities. Furthermore, multiple wired and/or wireless protocols maybe used, such as “3G” or “4G” data connectivity using a cellularcommunication protocol (such as CDMA, GSM, or WiMAX, as well as “WiFi”connectivity using IEEE 802.11).

The robotic system 100 may include access points through which thecontrol system 118 may communicate with a cloud server or other networkresource. Access points may take various forms such as the form of awireless access point (WAP) or wireless router. Further, if a connectionis made using a cellular air-interface protocol, such as a CDMA or GSMprotocol, an access point may be a base station in a cellular networkthat provides Internet connectivity via the cellular network. Otherexamples are also possible.

Further, the control system 118 may serve as an interface between therobotic system 100 and a user. For instance, the control system 118 mayinclude various components for communicating with the robotic system100. The example interfaces and communications may be implemented via awired or wireless connection, or both.

The control system 118 may include an input/output unit. Theinput/output unit may output information to a user through a display.The display may take on any form and may be arranged to project imagesand/or graphics to a user of the control system 118. As such, thedisplay may show a graphical user interface (GUI) that may provide anapplication through which the user may interact with the systemsdisclosed herein.

Additionally, the input/output unit may receive user-input (e.g., fromthe user of the control system 118). In particular, the input/outputunit may allow for interaction with the GUI such as for scrolling,providing text, and/or selecting various features of the application,among other possible interactions. The input/output unit may take onvarious forms. In one example, the input/output unit may include apointing device such as a computing mouse used for control of the GUI.However, if the input/output unit includes a touch screen display,touch-input can be received (e.g., such as using a finger or a stylus)that allows for control of the GUI. In another example, the input/outputunit may include a keyboard that provides for selection of numbers,characters and/or symbols to be displayed via the GUI. For instance, inthe arrangement where the input/output unit includes a touch screendisplay, portions the display may show the keyboard. Thus, touch-inputon the portion of the display including the keyboard may result inuser-input such as selection of specific numbers, characters, and/orsymbols to be shown on the GUI through the display. In yet anotherexample, the input/output unit may include a voice input device thatreceives audio input, such as from a user through a microphone, that isthen interpretable using one of various speech recognition techniquesinto one or more characters that may be shown through the display. Otherexamples may also be possible.

Operations of the control system 118 may be carried out by theprocessor(s) 102. Alternatively, these operations may be carried out bythe controller 108, or a combination of the processor(s) 102 and thecontroller 108. In some implementations, the control system 118 maypartially or wholly reside on a device other than the robotic system100, and therefore may at least in part control the robotic system 100remotely.

Mechanical components 110 represent hardware of the robotic system 100that enable the robotic system 100 to perform physical operations. As afew examples, the mechanical components 110 may include physical memberssuch as leg(s), arm(s), and/or wheel(s).

The physical members or other parts of robotic system 100 may furtherinclude actuators arranged to move the physical members in relation toone another. An actuator is a mechanism that may be used to introducemechanical motion. An actuator may be configured to convert storedenergy into movement of one or more components. Various mechanisms maybe used to power an actuator. For instance, actuators may be powered bychemicals, compressed air, hydraulics, or electricity, among otherpossibilities. With this arrangement, actuators may cause movement ofvarious movable components of the robotic system 100. For instance, theleg(s) and/or arm(s) may include one or more members connected by jointsand configured to operate with various degrees of freedom with respectto one another via actuators.

Furthermore, the mechanical components 110 may include one or moreend-effector(s). For instance, an end-effector may be disposed on theend of an arm, which can be operated via actuators to position theend-effector. The end-effector(s) may be employed to perform a task bymanipulating objects, acting on loads, etc. For instance, the roboticsystem 100 may use the end-effector(s) for gripping, turning, carrying,pulling, and/or pushing objects. The end-effector(s) may includehand-like structures with movable fingers. Alternatively oradditionally, the end-effector(s) may include other types of appendagesor attachments, such as grippers, welding tools, cutting tools, amongother possibilities.

The robotic system 100 may also include one or more structured bodiesfor housing the control system 118 and/or other components, and mayfurther include other types of mechanical components. The particularmechanical components 110 used in a given robot may vary based on thedesign of the robot, and may also be based on the operations and/ortasks the robot may be configured to perform.

In some examples, the mechanical components 110 may include one or moreremovable components. The robotic system 100 may be configured to addand/or remove such removable components, which may involve assistancefrom a user and/or another robot. For example, the robotic system 100may be configured with removable arms, hands, feet, legs, and/orend-effectors so that these appendages can be replaced or changed asneeded or desired. In some embodiments, the robotic system 100 mayinclude one or more removable and/or replaceable battery units orsensors. Other types of removable components may be included within someimplementations.

The robotic system 100 may include the sensor(s) 112 arranged to senseaspects of the robotic system 100. The sensor(s) 112 may include one ormore force sensors, torque sensors, velocity sensors, accelerationsensors, gyroscopic sensors, position sensors, proximity sensors, motionsensors, location sensors, load sensors, temperature sensors, thermalimaging sensors, touch sensors (e.g., capacitive sensors), opticalsensors, wireless sensors, radio sensors, depth sensors (e.g., RGB-D,laser, structured-light, and/or a time-of-flight camera), point cloudsensors, range sensors (e.g., ultrasonic and/or infrared), infraredsensors, object sensors, and/or cameras (e.g., two-dimensional (2D)cameras, color cameras, grayscale cameras, and/or infrared cameras),three-dimensional (3D) scanners, x-ray scanners, spectral scanners,Radio Frequency identification (RFID) systems, Near Field Communication(NFC) chips, among other possibilities. Within some examples, therobotic system 100 may be configured to receive sensor data from thesensor(s) 112 that are physically separated from the robot (e.g.,sensors that are positioned on other robots or located within theenvironment in which the robot is operating). Further, the sensor(s) 112may be incorporated within existing devices, such as mobile phones,laptops, and/or tablets.

The sensor(s) 112 may provide sensor data to the processor(s) 102(perhaps by way of data 107) to allow for interaction of the roboticsystem 100 with its environment, as well as monitoring of the operationof the robotic system 100. The sensor data may be used in evaluation ofvarious factors for activation, movement, and deactivation of mechanicalcomponents 110 and electrical components 116 by control system 118. Forexample, the sensor(s) 112 may capture data corresponding to the terrainof the environment or location of nearby objects, which may assist withenvironment recognition and navigation. In an example configuration,sensor(s) 112 may include RADAR (e.g., for long-range object detection,distance determination, and/or speed determination), LIDAR (e.g., forshort-range object detection, distance determination, and/or speeddetermination), SONAR (e.g., for underwater object detection, distancedetermination, and/or speed determination), VICON® (e.g., for motioncapture), a laser tracker system, one or more cameras (e.g.,stereoscopic cameras for 3D vision), a global positioning system (GPS)transceiver, and/or other sensors for capturing information of theenvironment in which the robotic system 100 is operating. The sensor(s)may also include The sensor(s) 112 may monitor the environment in realtime, and detect obstacles, elements of the terrain, weather conditions,temperature, and/or other aspects of the environment.

Further, the robotic system 100 may include sensor(s) 112 configured toreceive information indicative of the state of the robotic system 100,including sensor(s) 112 that may monitor the state of the variouscomponents of the robotic system 100. The sensor(s) 112 may measureactivity of systems of the robotic system 100 and receive informationbased on the operation of the various features of the robotic system100, such the operation of extendable legs, arms, or other mechanicaland/or electrical features of the robotic system 100. The data providedby the sensor(s) 112 may enable the control system 118 to determineerrors in operation as well as monitor overall operation of componentsof the robotic system 100.

As an example, the robotic system 100 may use force sensors to measureload on various components of the robotic system 100. In someimplementations, the robotic system 100 may include one or more forcesensors on an arm or a leg to measure the load on the actuators thatmove one or more members of the arm or leg. As another example, therobotic system 100 may use one or more position sensors to sense theposition of the actuators of the robotic system. For instance, suchposition sensors may sense states of extension, retraction, or rotationof the actuators on arms or legs.

As another example, the sensor(s) 112 may include one or more velocityand/or acceleration sensors. For instance, the sensor(s) 112 may includean inertial measurement unit (IMU). The IMU may sense velocity andacceleration in the world frame, with respect to the gravity vector. Thevelocity and acceleration sensed by the IMU may then be translated tothat of the robotic system 100 based on the location of the IMU in therobotic system 100 and the kinematics of the robotic system 100.

The robotic system 100 may include other types of sensors not explicateddiscussed herein. Additionally or alternatively, the robotic system mayuse particular sensors for purposes not enumerated herein.

The robotic system 100 may also include one or more power source(s) 114configured to supply power to various components of the robotic system100. Among other possible power systems, the robotic system 100 mayinclude a hydraulic system, electrical system, batteries, and/or othertypes of power systems. As an example illustration, the robotic system100 may include one or more batteries configured to provide charge tocomponents of the robotic system 100. Some of the mechanical components110 and/or the electrical components 116 may each connect to a differentpower source, may be powered by the same power source, or be powered bymultiple power sources.

Any type of power source may be used to power the robotic system 100,such as electrical power or a gasoline engine. Additionally oralternatively, the robotic system 100 may include a hydraulic systemconfigured to provide power to the mechanical components 110 using fluidpower. Components of the robotic system 100 may operate based onhydraulic fluid being transmitted throughout the hydraulic system tovarious hydraulic motors and hydraulic cylinders, for example. Thehydraulic system may transfer hydraulic power by way of pressurizedhydraulic fluid through tubes, flexible hoses, or other links betweencomponents of the robotic system 100. The power source(s) 114 may chargeusing various types of charging, such as wired connections to an outsidepower source, wireless charging, combustion, or other examples.

The electrical components 116 may include various mechanisms capable ofprocessing, transferring, and/or providing electrical charge or electricsignals. Among possible examples, the electrical components 116 mayinclude electrical wires, circuitry, and/or wireless communicationtransmitters and receivers to enable operations of the robotic system100. The electrical components 116 may interwork with the mechanicalcomponents 110 to enable the robotic system 100 to perform variousoperations. The electrical components 116 may be configured to providepower from the power source(s) 114 to the various mechanical components110, for example. Further, the robotic system 100 may include electricmotors. Other examples of electrical components 116 may exist as well.

The robotic system 100 may include a body, which may connect to or houseappendages and components of the robotic system. As such, the structureof the body may vary within examples and may further depend onparticular operations that a given robot may have been designed toperform. For example, a robot developed to carry heavy loads may have awide body that enables placement of the load. Similarly, a robotdesigned to reach high speeds may have a narrow, small body that doesnot have substantial weight. Further, the body and/or the othercomponents may be developed using various types of materials, such asmetals or plastics. Within other examples, a robot may have a body witha different structure or made of various types of materials.

The body and/or the other components may include or carry the sensor(s)112. These sensors may be positioned in various locations on the roboticdevice 100, such as on the body and/or on one or more of the appendages,among other examples.

On its body, the robotic device 100 may carry a load, such as a type ofcargo that is to be transported. The load may also represent externalbatteries or other types of power sources (e.g., solar panels) that therobotic device 100 may utilize. Carrying the load represents one exampleuse for which the robotic device 100 may be configured, but the roboticdevice 100 may be configured to perform other operations as well.

III. EXAMPLE EMBODIMENT OF A ROBOTIC SYSTEM FOR PRODUCING A STRUCTURE

FIG. 2 illustrates a robot 200 that includes aspects of the roboticsystem 100 described above, such as the mechanical components 110, thesensor(s) 112, the power source(s) 114, the electrical components 116,and/or the control system 118. Operating at a production site 26, therobot 200 works with a plurality of physical objects 22 to produce aphysical structure 20.

According to one example, a production site may be a factory floor wherea robot installs parts in an assembly line to build a product (e.g., atable, airplane wing, etc.). According to an additional example, aproduction site may be a work cell, rather than an assembly line, wherea robot combines a variety of parts to build a product from start tofinish. In these examples, the production site may be a temporarylocation from which the final physical structure may be delivered (e.g.,as a product) to another location (e.g., a distributor or customerlocation) when completely built.

According to another example, a production site may be a municipal sitewhere a robot works with heavy construction materials to construct abridge or a road. According to a further example, a production site maybe a construction site where a robot works with construction materialsto construct a house or a building. According to yet another example, aproduction site may be the interior of a house where a robot installshousing materials to construct a section of the house. In theseexamples, the final physical structure is installed in the productionsite.

As shown in FIG. 2. the robot 200 is configured to complete specifictasks at the production site 26. For instance, the robot 200 may includea variety of end-effectors 211. In particular, an end-effector 211(a) isdisposed at the end of an arm 210 a that allows the robot 200 to grip aphysical object 22 a and position the physical object 22 a at aspecified location 20 a at the production site 26. The robot 200 mayalso include another end-effector 211(b) disposed at the end of anotherarm 210 b that allows the robot 200 to fix or otherwise couple thephysical object 22 a to a surface at the specified location 20 a. Forinstance, the end-effector 211 b may include a tool, such as a nail gun,a glue gun, or welding torch, to fix the physical object 22 a in place.

Although FIG. 2 illustrates the single robot 200, the production site 26may include other robots that work with the robot 200 to produce aspectsof the physical structure 20. Each robot at the production site 26 maybe configured to complete specific assigned tasks in the productionprocess. Thus, in an alternative example, the robot 200 may be assignedto position the physical object 22 a in the specified location 20 a,while another robot (not shown) may be assigned to fix the physicalobject 22 a to the specified location 20 a with its own end-effector(e.g., nail gun, glue gun, or welding torch).

As shown in FIG. 2, the robot 200 may employ various sensors 212 toguide its operation at the production site 26. The sensors 212 mayinclude local sensors that are directly coupled to the robot 200.Additionally or alternatively, the sensors 212 may include globalsensors that are disposed in other areas of the production site 26separate from the robot 200. For instance, some of the global sensorsmay be arranged in fixed locations throughout the production site 26.Additionally or alternatively, some of the global sensors may be coupledto other robots at the production site 26.

The sensors 212 may include any of the sensors 112 described above. Forinstance, to produce the physical structure 20, one or more sensors212(a) can determine the location of the robot 200 relative to a storagelocation 20 b where the physical objects 22 are stored at the productionsite 26. The robot 200 can employ data from the one or more sensors212(a) to move to the location 20 b and place the physical objects 22within range of the end-effector 211(a). Additionally, one or moresensors 212(b) can determine the location of the end-effector 211(a)relative to the physical objects 22, so that the arm 210 a can beextended to the retrieve the physical object 22 a from the storagelocation 20 b with the end-effector 211(a). With the physical object 22a in the grip of the end-effector 211(a), the one or more sensors 212(a)can then determine the location of the robot 200 relative to thelocation 20 a. The robot 200 can employ data from the one or moresensors 212(a) to move to the position 20 a where the physical object 22a is to be fixed in the production process. The one or more sensors212(b) can then determine the location of the end-effector 211(a)relative to the location 20 a, so that the arm 210 a can be extended toposition the physical object 22 a precisely at the location 20 a withthe end-effector 211(a). As shown in FIG. 2, the one or more sensors212(a) may be global sensors, while the one or more sensors 212(b) maybe local sensors disposed on the arm 210 b of the robot 200. The sensors212(a), (b), however, may be any combination of local and globalsensors.

One or more additional sensors 212 may be further employed so that thephysical object 22 a is manipulated and positioned at the location 22 awith a desired orientation. Other sensors 212 may be employed to guidethe operation of the end-effector 211(b) to fix the physical object 22 ato a surface at the location 20 a. Yet other sensors 212 may performmeasurements to determine the size and shape of the physical object 22 aso that the physical object 22 a can be positioned at the location 20 awith the appropriate orientation.

A control system 218 is employed to operate the robot 200. The controlsystem 218 may be similar to the control system 118 described above.Aspects of the control system 218 may be included as part of the robot200 and/or be separate from the robot 200.

The control system 218 can receive data from the sensors 212 via wiredand/or wireless connections. The control system 218 can then operate therobot 200 in response to the sensor data. For instance, in the exampleabove, the control system 218 can cause the robot 200 to move tolocations 20 a, b based on the data from the sensors 212(a) and tomanipulate or otherwise engage the physical object 22 a based on thedata from the sensors 212(b).

IV. EXAMPLE MODEL FOR PRODUCING A STRUCTURE WITH A ROBOTIC SYSTEM

Referring to FIG. 3A, the control system 218 of the robot 200 above isguided by specifications 310 based on a model 300 to produce thephysical structure 20 from the plurality of physical objects 22. Inparticular, the specifications 310 may be embodied as computer-readableprogram instructions stored on data storage (e.g., data storage 104) ofthe control system 218. A designer develops the model 300 for thephysical structure 20. Generally, the model 300 provides the designer'sconception of how the physical structure 20 will be configured and howit will perform when actually produced by the robot 200.

To develop the model 300, the designer defines a plurality of modelobjects 302. In general, the model objects 302 provide the designer'sconception of what physical objects 22 are combined to produce thephysical structure 20 according to the model 300. The designer designseach model object 302 to contribute an appropriate set of properties 303that helps the physical structure 20 to provide the desiredconfiguration and performance. Some of the model objects 302 may sharean identical set of properties 303, while other model objects 302 mayhave different respective sets of properties 303.

The set of properties 303 of each model object 302 may indicate desiredaesthetic properties, such as color, markings, visual patterns, shape,size, and surface finish/texture, among other properties. Additionallyor alternatively, the set of properties 303 may indicate desiredmechanical properties, such as bending strength, brittleness, bulkmodulus, coefficient of friction, compressive strength, creep,elasticity, fatigue strength, flexibility, fracture toughness, hardness,plasticity, resilience, shear strength, stiffness, stress/strainproperties, surface roughness, tensile strength, toughness, viscosity,yield strength, and weight, among other properties. Additionally oralternatively, the set of properties 303 may indicate electrical and/ormagnetic properties, such as capacitance, conductivity, density,dielectric strength, field properties, inductance, permittivity, andresistance, among other properties. Additionally or alternatively, theset of properties 303 may indicate chemical properties, such ascorrosion resistance, flammability, pH, reactivity, stability, surfaceenergy/tension, and toxicity, among other properties. Additionally oralternatively, the set of properties 303 may indicate manufacturingproperties for coating, cutting, drilling, forming and shapingprocesses, heat treating, joining, machining, rolling, sanding, andwelding, among other techniques. Additionally or alternatively, the setof properties 303 may indicate optical properties, such as absorbance,fluorescence, photosensitivity, reflectivity, refractive index,scattering, and transmittance, among other properties. Additionally oralternatively, the set of properties 303 may indicate thermalproperties, such as boiling point, critical point, emissivity, meltingpoint, specific heat, thermal conductivity, thermal diffusivity, andthermal expansion, among other properties.

In some cases, a model object 302 may refer to a stock part with a knownset of properties 303. As such, specifying the stock part may besufficient to communicate the set of properties 303, and it may not benecessary to state the entire set of properties 303 expressly whendeveloping the model 300.

In other cases, a model object 302 may refer to a part that is formedfrom stock material with a known set of properties 303. For instance,the model part 302 may be a rod of stock material that is cut to acertain length. As such, specifying the stock material and indicatingspecific dimensions for the model object 302 may be sufficient tocommunicate the set of properties 303, and it may not be necessary tostate the properties of the stock material expressly when developing themodel 300.

Although some model objects 302 may model raw materials or simpleobjects with fewer parts, other model objects 302 may also model morecomplex sub-assemblies. Each sub-assembly combines a plurality of parts,which act together to provide a desired set of properties. For instance,a sub-assembly may provide a motor, where the motor itself includes aplurality of different parts that work together to electromechanicallydrive other physical objects 22 in the physical structure 20. In thiscase, the set of properties for the motor may include size, shape, poweroutput, and torque output, among other properties.

To develop the model 300, the designer also defines relationships 304between each model object 302 and other model objects 302. Therelationships 304 determine how the model objects 302 are combined. Forinstance, the designer may determine positions/orientations of the modelobjects 302 relative to other model objects 302. In addition, thedesigner may determine how the model objects 302 are directly orindirectly coupled to each other. Such couplings may include mechanicalcouplings, electrical/electronic couplings, or data communicationcouplings, among other possibilities. Furthermore, the designer maydetermine what manufacturing, assembly, or other production techniquesto employ to combine the model objects 302.

The designer may further define aspects of the model 300 according to amodel production site 306. The model production site 306 models externalstructures and conditions at the production site 26, which are relevantto producing the physical structure 20.

The designer determines relationships 308 between the model objects 302and the model production site 306. For instance, the designer maydetermine positions/orientations of the model objects 302 relative toexternal structures in the model production site 306. In addition, thedesigner may determine how the model objects 302 are coupled to externalstructures in the model production site 306. According to one example,the model objects 302 may need to be shaped and sized so that they canbe installed onto external structures in the model production site 306.

Furthermore, the designer may determine how the model objects 302 areaffected by external conditions in the model production site 306.According to another example, the model objects 302 may need to beconfigured to withstand atmospheric conditions (e.g., heat, humidity,etc.) in the model production site 306.

The model 300 is translated into the specifications 310, which guide theproduction of the physical structure 20 by the robot 200 according tothe designer's conception. In some cases, the designer may develop themodel 300 via computer-implemented software that receives input relatingto the model 300 and generates the specifications 310 for the robot 200from this input. For instance, the input may include the definition ofthe model objects 302, the relationships 304 between model objects 302,and the relationships 308 between the model objects 302 and the modelproduction site 306. The designer may employ computer-aided design (CAD)tools in the software to specify aspects of the model 300, e.g.,relative position, angular offset (e.g., perpendicular, parallel), axialalignment, mating, etc., of the model objects 302.

As described above, the specifications 310 may be embodied ascomputer-readable program instructions stored on data storage (e.g.,data storage 104) of the control system 218. The control system 218 canexecute the computer-readable program instructions to operate aspects ofthe robot 200 to build the physical structure 20.

Accordingly, the physical objects 22, which are modeled by the modelobjects 302, are combined by the robot 200 to produce the physicalstructure 20 according to the model 300. In addition, the robot 200produces the physical structure 20 according to external structures andconditions at the production site 26, which is modeled by the modelproduction site 306. According to one example, the model production site306 models an interior section of a house where housing materials areinstalled according to the model 300. According to another example, themodel production site 306 models a municipal site where a bridge or aroad is built according to the model 300.

FIGS. 4A-B illustrate aspects of an example model 400 for installingphysical tiles 42 on a physical wall 47, as conceived by a designer. Inthe model 400, model tiles 402 are mounted on a model wall 407. Themodel tiles 402 model physical tiles 42 for producing a physicalstructure, i.e., a tile installation 40. Meanwhile, the model wall 407is an external structure in a model production site 406, which forinstance may model an interior 46 of a house.

The designer defines the model tiles 402 to have a desired set ofproperties 403. For instance, the model tiles 402 are all uniformly madefrom the same material as rectangular solid objects having the sameheight h, width w, and thickness t. Additionally, the model tiles 402each have a vertical front surface 402 a that provides desired aestheticproperties, such as surface finish(es), color(s), design pattern(s),etc.

The designer determines relationships 404 between the model tiles 402.For instance, the model tiles 402 are arranged in a pattern of five rowsR_(i=1, 2, 3, 4, 5) and five columns C_(i=1, 2, 3, 4, 5), where each rowR_(i) has five model tiles 402 spaced uniformly apart by a distance dand each column C_(i) includes five model tiles 402 spaced uniformlyapart by the same distance d.

The designer also determines relationships 408 between the model tiles402 and external structures and conditions of the model production site406. The designer identifies a set of properties for the model wall 407.For instance, the model wall 407 includes a vertical front surface 407 awith a rectangular area A for receiving the model tiles 402. Therectangular area A is defined by a top edge E_(TOP), a bottom edgeE_(BOTTOM), a right edge E_(RIGHT), and a left edge E_(LEFT), Thepattern of model tiles 402 is arranged within the rectangular area A.Furthermore, as shown in FIG. 4A, a top row R₁ of model tiles 402 abutsthe top edge E_(TOP) of the rectangular area A, a bottom row R₅ of modeltiles 402 abuts the bottom edge E_(BOTTOM) of the rectangular area A, aleft column C₁ of model tiles 402 abuts the left edge E_(LEFT) of therectangular area A, and a right column C₅ of model tiles 402 abuts theright edge E_(RIGHT) of the rectangular area A.

As described above, the designer may develop the model 400 viacomputer-implemented software that receives input relating to the model400. For instance, the designer can employ such software to define themodel tiles 402 and specify the relationships 404 between the modeltiles 402 and the relationships 408 between the model tiles 402 and theproduction site 406.

According to the model 400, the model tiles 402 are mounted onto thefront surface 407 a of the model wall 407 with mortar. Each model tile402 has a back surface 402 b with a surface texture that bondseffectively with the mortar. Correspondingly, the front surface 407 ahas a surface texture that also bonds with the mortar. Additionally, thespace d between the model tiles 402 is filled with tile grout. Theconditions in the model production site 406 allow the mortar and thegrout to cure properly. For instance, the atmospheric conditions in themodel production site 406 provide the appropriate temperature andhumidity for use of the mortar.

Accordingly, as shown in FIG. 4B, the model 400 identifies a pluralityof model objects, i.e., the model tiles 402, and indicates how theplurality of model objects are combined to produce a physical structure,i.e., the tile installation 40. Furthermore, the model 400 identifies anexternal structure, i.e., the model wall 407, and external conditions,e.g., temperature and humidity, that are relevant to combining the modelobjects 402. The model 400 is translated into specifications 410, whichguide the production of the tile installation 40 by the robot 200according to the designer's conception. In particular, thespecifications 410 may be embodied as computer-readable programinstructions stored on data storage (e.g., data storage 104) of thecontrol system 218.

The properties 403 of the model tiles 402, the relationships 404 betweenthe model tiles 402, the model production site 406, and therelationships 408 between the model tiles 402 and the model productionsite 406 establish a variety of parameters that determine how the tileinstallation 40 can be produced. By providing particular values,settings, and other instructions for these parameters, thespecifications 410 provide the robot 200 with guidance on the productionof the tile installation 40.

The control system 218 of the robot 200 can execute thecomputer-readable program instructions to operate aspects of the robot200 to build the tile installation 40. Thus, the physical tiles 42,which are modeled by the model tiles 402, are combined by the robot 200to produce the tile installation 40 according to the model 400.

V. EXAMPLE EMBODIMENT OF ROBOTIC SYSTEM FOR PRODUCING A STRUCTURE

As described above with reference to FIG. 3A, a designer can develop amodel 300 to provide the specifications 310 for producing the physicalstructure 20 with the robot 200. The model 300, though, might onlyreflect how the physical structure 20 will be theoretically produced bythe robot 200 as conceived by the designer. For instance, the model 300might assume that the model objects 302 are accurately positioned andoriented relative to each other. In reality, however, the productionprocess might introduce variations or errors by positioning andorienting each physical object 22 with varying degrees of accuracy. Evenif each variation/error on its own may appear to be minor, the accrualof variations/errors may prevent the production process from producingthe final physical structure 20 as intended by the designer.

When developing the model 300, the designer may attempt to make theproduction process for the physical structure 20 more robust byproviding tolerances for acceptable variations introduced by theproduction process. For instance, the model 300 may include tolerancesfor small variations in the position or orientation of each physicalobject 22.

Even when the robot 200 installs the physical objects 22 with variationswithin the tolerances, however, such variations may accumulate to createan unanticipated effect on the final physical structure 20. Furthermore,the robot 200 may install one or more of the physical objects 22 witherrors outside of the tolerances. In general, the robot 200 mayencounter production issues that cannot be sufficiently addressed by thetolerances.

Advantageously, as shown in FIG. 3B, the robot 200 can continuouslymonitor the production process. As the robot 200 installs of eachphysical object 22 for instance with end-effector(s) 211, the robot 200can employ any of the sensors 212 to measure any variations/errors 314introduced by the installation of each physical object 22. For instance,the robot 200 can employ a 2D camera, 3D scanner, x-ray scanner,ultrasound sensor, spectral scanner, load sensors, etc., to take variousmeasurements of aspects of the physical structure 20 and/or theproduction site 26. The control system 218 can use the measurements asfeedback to identify any variations/errors 314. The control system 218can then determine the effect of the variations/errors 314 on the finalphysical structure 20. Furthermore, the control system 218 can determineadjustments 316 for the production process to mitigate the effect of thevariations/errors 314.

As shown in FIG. 3B, the adjustments 316 are applied to thespecifications 310 to provide the customized production process for thephysical structure 20. In general, parameters of the model 300 havemodifiable values, settings, etc. As such, the specifications 310provide a parameter space that allows for the adjustments 316. In otherwords, the control system 218 can determine the adjustments 316 byselecting values, settings, etc., for one or more of the parameters ofthe model 300 in response to the variations/errors 314. The selection ofthese values, settings, etc., can optimize the parameter space for themodel 300. Once the control system 218 determines the adjustments 316,the control system 218 further controls the robot 200, e.g., theend-effector(s) 211, to proceed with the production process according tothe adjustments 316. The robot 200 continues to monitor the productionprocess according to the feedback loop shown in FIG. 3B. By allowing therobot 200 to respond to the effect of the variations/errors 314 on itsown, the production process can continue without interruption and/orintervention by operators or the like.

Although the adjustments 316 to the production process may changeaspects of the resulting physical structure 20, the robot 200 operatesso that the production process preserves the intent of the designer asset forth in the model 300. In other words, the robot 200 can customizethe production process to address issues in the production process whilestill producing a physical structure 20 generally consistent with themodel 300.

When developing the model 300, the designer may define one or moreconstraints 311 that establish the scope of allowable adjustments 316 tothe production process. In particular, the constraints 311 establish theparameter space described above and the adjustments 316 include values,settings, etc., for parameters in that space. The constraints 311 mayinclude tolerances 313 as described above.

Furthermore, the constraints 311 provide guidance for determining theadjustments 316. The constraints 311 generally reflect the features thatare most significant to the designer and indicate the designer's intentbehind the model 300. In some cases, constraints 311 may also bedetermined according to other considerations, such as safety regulations(e.g., for the production of an airplane wing).

Referring to FIGS. 3A-B, the specifications 310 communicated to therobot 200 also include information on the constraints 311. The robot200, via the control system 218, can evaluate the constraints 311 todetermine how to customize the production process for the physicalstructure 20. In general, the robot 200 has the flexibility to customizethe production process as long as the physical structure 20 satisfiesthe constraints 311.

When developing the model 300, the designer may also define weightings312 for the constraints 311. With the weightings 312, the specifications310 can further guide customization of the production process. The robot200 can evaluate the weightings 312 to determine a preferred approachthat yields a physical structure 24 most consistent with the designer'sintent. For instance, the constraints 311 may be weighted to indicatethe relative importance that the designer places on each constraint 311.The weightings 312 may indicate that some constraints 311 mustnecessarily be satisfied to allow the production process to proceed.Meanwhile, the weightings 312 may indicate that other constraints 312are preferences that may be modified (or even ignored) in order toproduce the desired physical structure. For instance, some constraints312 may specify preferred values, settings, or other instructions forparameters in the production process, but if necessary, such parametersmay be adjusted to respond to feedback during the production process.Furthermore, the weightings 312 may indicate the order in whichmodifiable constraints 311 should be modified.

As described above, the designer may develop the model 300 viacomputer-implemented software that receives input relating to the model300. As such, the designer may specify the constraints 311, includingthe weightings 312 and the tolerances 313 via the software, which inturn generates the specifications 310, in the form of computer-readableprogram instructions, for the robot 200.

After evaluating a variation/error resulting from the installation ofone of the physical objects 22, the robot 200 might determine that oneor more constraints 311 cannot be satisfied even if the productionprocess is modified. In such cases, the robot 200 can abort theproduction process. In other words, the robot 200 can make a “go/no go”decision after the installation of each physical object 22. If the robot200 makes a “no go” decision, the robot 200 can communicate an alert, sothat a user (or other system) can intervene and determine how toproceed. If possible, further information and/or instructions arecommunicated to the robot 200 to allow the robot 200 to proceed with theproduction process. For instance, one or more of the the constraints 311may be modified for adjustments 316 to the production process.

The robot 200 may automatically document the progress of the productionprocess as it evaluates the installation of each physical object 22 andmakes adjustments 316 to address any variations/errors 314 during theinstallation. For instance, the robot 200 may record any measurements itmakes with the sensors 212 after the installation of each physicalobject 22 as well as any adjustments made in response. Suchdocumentation may be saved and/or communicated to another party orsystem to provide a real time (or substantially real time) log as theproduction process progresses. After the production process is complete,the final documentation may be employed to provide detailed information(e.g., measurements) on the entire physical structure 24 as well as areport on how the production process was adjusted in response to thevariations/errors 314.

As described above, FIGS. 4A-B illustrate the example model 400 forproducing the tile installation 40 (physical structure) on the wall 41(external structure) in a house interior 46 (production site). The model400 is translated into the specifications 410, which can guide theproduction of the tile installation 40 by the robot 200. As shownparticularly in FIG. 4B, the specifications 410 may provide informationrelating to constraints 411, including weightings 412 and tolerances413.

FIG. 5A illustrates example specifications 410, including information onexample constraints 411 a-f, based on the model 400. In particular, afirst constraint 411 a requires the physical tiles 42, prior toinstallation, to be rectangular solids with substantially the height h₀,the width w₀, and the thickness t₀. A second constraint 411 b requiresthe physical wall 47 to have a front surface 47 a with a rectangulararea A across which the physical tiles 42 can be mounted. Additionally,a third constraint 411 c requires the physical tiles 42 to be mountedaccording to a pattern of spaced rows R_(i) and spaced columns C_(i),where each row R_(i) has five physical tiles 42 and each column C_(i)includes five physical tiles 42. Furthermore, a fourth constraint 411 drequires the pattern of physical tiles 42 to abut a top edge E_(TOP), abottom edge E_(BOTTOM), a left edge E_(LEFT), and a right edge E_(RIGHT)of the rectangular area A.

As shown in FIG. 5A, the specifications 410 also include information onweightings 412 a-d for the respective constraints 411 a-d. In thisparticular example, the weightings 412 a-d indicate the highestimportance for the constraints 411 a-d and require the tile installation40 to meet the constraints 411 a-d.

The specifications 410 also include information on a fifth constraint411 e. The fifth constraint 411 e calls for the five physical tiles 42of each row R₁ to be spaced uniformly apart at a distance of D_(R) witha tolerance 413 e _(R) of +/−δ_(R) and the five physical tiles 42 ofeach column C₁ to be spaced uniformly apart at a distance of D_(C) witha tolerance 413 e _(C) of +/−δ_(C). The fifth constraint 411 e isassociated with a weighting 412 e. According to the weighting 412 e, themodel 400 places a lower priority on the fifth constraint 411 e than theconstraints 411 a-d. In other words, aspects of the tile installationprocess, if necessary, can modify the fifth constraint 411 e as long asthe tile installation 40 can satisfy the constraints 411 a-d. Becauseaspects of the fifth constraint 412 e are modifiable, the fifthconstraint 412 e defines a part of the parameter space for the model400. Here, the modifiable parameters include the spacing between thephysical tiles 42 of each row R_(i) and the spacing between the physicaltiles 42 for each column C_(i). The fifth constraint 411 e specifies apreferred value of D_(R)+/−δ_(R) for the spacing in each row R_(i) and apreferred value of D_(C)+/−δ_(C) for the spacing in each row C_(i).However, the values can be modified, if necessary, to determineadjustments 416 for the tile installation process.

In operation, the robot 200 can maneuver itself at the production site(i.e., the house interior 46) and use any of the end-effectors 211 andthe sensors 212 described above to manipulate and mount the physicaltiles 42 on the physical wall 47. Correspondingly, the robot 200 canemploy the appropriate sensor(s) 212 (e.g., 2D camera, 3D scanner, x-rayscanner, ultrasound sensor, spectral scanner, load sensors, etc.) tomeasure any variation/error that may result from each mounting of thephysical tiles 42. Additionally, the robot 200, via the control system218, can determine whether the constraints 411 can still be satisfied.If necessary, the robot 200 can determine, via the control system 218,what aspects of the tile installation process may need to be allowablymodified to respond to the variation/error. Aspects of the feedback loopshown in FIG. 3B apply to this example.

FIGS. 5B-D illustrate example steps in a process for producing the tileinstallation 40 according to the specifications 410. Prior to installingthe physical tiles 42, the robot 200 employs the appropriate sensor(s)212 to measure the physical tiles 42 as well as the rectangular area Aof the wall 47. In this example, the physical tiles 42 are allrectangular solids with substantially the height h₀, the width w₀, andthe thickness t₀. Additionally, the rectangular area A has a height of5h₀+4D_(C)+2δ_(C) and a width of 5w₀+4D_(R).

As shown in FIG. 5B, the robot 200 mounts a second physical tile 42 b onthe physical wall 47 after the installation of a first physical tile 42a. The physical tiles 42 a, b are aligned to form part of a left columnC₁. After mounting the second tile 42 b, the robot 200 employs theappropriate sensor(s) 212 to measure the spacing between the firstphysical tile 42 a and the second physical tile 42 b. The secondphysical tile 42 b is mounted a distance of D_(C)+v below the firstphysical tile 42 a. In this case, the value of v happens to be equal toδ_(C), which corresponds to the tolerance 413 e _(C) provided by thefifth constraint 411 e. As such, the value of v falls within a variationanticipated by the designer for the mounting of the physical tiles 42.(To simplify this discussion, the value of the variation v here equalsδ_(C), but in other cases, the value of v may be less than δ_(C).)

Using the spacing measurement obtained by the robot 200, the controlsystem 218 evaluates whether three additional physical tiles 42 can bemounted below the second physical tile 42 b, where the left column C₁abuts the bottom edge E_(BOTTOM) of the rectangular area A according tothe fourth constraint 411 d.

The first physical tile 42 a and the second physical tile 42 b combinefor a height of 2h₀+D_(C)+v=2h₀+D_(C)+δ_(C). As the rectangular area Ahas a height of 5h₀+4D_(C)+2δ_(C), the rectangular area A provides afurther 3h₀+3D_(C)+δ_(C) to accommodate three additional physical tiles42 to complete the left column C₁. Even with the variation v in thespacing between the first physical tile 42 a and the second physicaltile 42 b, the control system 218 determines that the height of therectangular area A is sufficient to allow the mounting of the threeadditional physical tiles 42. Thus, the control system 218 determinesthat the robot 200 can proceed with the tile installation process andstill satisfy the constraints 411 a-e without any adjustments 416 atthis stage.

As shown in FIG. 5C, the robot 200 mounts a third physical tile 42 c onthe physical wall 47 after the installation of the physical tiles 42 a,42 b. The physical tile 42 c is aligned with the physical tiles 42 a-bto form part of the left column C₁. After mounting the third physicaltile 42 c, the robot 200 employs the appropriate sensor(s) 212 tomeasure the spacing between the second physical tile 42 b and the thirdphysical tile 42 c. Although the robot 200 attempts to mount thephysical tiles 42 as precisely as possible according to the constraint511 e, the third physical tile 42 c is mounted with at a distance ofD_(C)+e₁ below the second physical tile 42 b, where e₁ is an error. Inthis case, the value of error e₁ is greater than δ_(C), whichcorresponds to the tolerance 413 e _(C) provided by the fifth constraint411 e. As such, the value of e₁ falls outside a variation anticipated bythe designer for the mounting of the physical tiles 42. Because thedistance D_(C)+e₁ is greater than the D_(C)+δ_(C), the mounting of thethird physical tile 42 c does not satisfy the fifth constraint 411 e.The weighting 412 e for the fifth constraint 411 e, however, indicatesthat the parameters of the fifth constraint 411 e, i.e., the spacingbetween the tiles, may be modified for the mounting of the thirdphysical tile 42 c to allow the tile installation process to proceeddespite the error e₁.

Using the spacing measurement obtained by the robot 200, the controlsystem 218 also evaluates whether two additional physical tiles 42 canstill be mounted to complete the left column C₁, where the left columnC₁ abuts the bottom edge E_(BOTTOM) of the rectangular area A accordingto the fourth constraint 411 d. The physical tiles 42 a-c combine for aheight of 3h₀+2D_(C)+v+e₁, which is greater than 3h₀+2D_(C)+2δ_(C). Asthe rectangular area A has a height of 5h₀+4D_(C)+2δ_(C), therectangular area A provides less than 2h₀+2D_(C) to accommodate twoadditional physical tiles 42 to complete the left column C₁. Thus, thecontrol system 218 determines that the remaining space in therectangular area A is now insufficient to allow the mounting of the twoadditional physical tiles 42 d, e (with the height h₀) while alsosatisfying the constraints 411 a-e, as such mounting requires a distanceof at least 2h₀+2D_(C).

As described above, the parameters of the fifth constraint 411 e, i.e.,the spacing between the tiles, may be modified for adjustments 316 tothe tile installation process. Therefore, in response to the error e₁ aswell as the variation v, the control system 218 may adjust the spacingsbetween the third physical tile 42 c and the two subsequent physicaltiles 42 d, e. In particular, the mounting of physical tiles 42 d, erequires 2h₀. Assuming that the rectangular area A provides at least 2h₀after the third physical tile 42 c is mounted, the control system 218may determine that the remaining space (less than 2D_(C)) can be dividedevenly for: (i) the spacing between the third physical tile 42 c and thefourth physical tile 42 d, and (ii) the spacing between the fourthphysical tile 42 d and the fifth physical tile 42 e.

If, however, the rectangular area A does not provide at least 2h₀ afterthe third physical tile 42 c is mounted, the control system 218determines that two additional tiles 42 cannot be mounted according tothe requirements of the third constraint 411 c. Thus, the control system218 can decide to abort the tile installation process after theinstallation of the third physical tile 42 c. Advantageously, the robot200 aborts the tile installation process before additional resources arewasted in the production of an unwanted tile installation. With thisdecision, the control system 218 can communicate an alert, so that auser (or other system) can intervene and determine how to proceed.

Each time the control system 218 receives feedback from the sensors 212,the control system 218 can respond to any variations/errors by(re)evaluating and adjusting the values, settings, etc., for the entireparameter space defined by the model 400. For instance, the adjustments416 after the mounting of a particular physical tile 42 may yield a setof spacings for the physical tiles 42 yet to be mounted, but once thesubsequent physical tile 42 is mounted, the control system 218 maydetermine, due to the introduction of further variations/errors, furtheradjustments 416 that yield a different set of spacings for the physicaltiles 42 yet to be mounted.

As shown in FIG. 5A, the specifications 410 include a sixth constraint411 f which provides that some of the physical tiles 42, if necessary,can be resized to fit the tile installation 40 into the remaining spaceprovided by the rectangular area A. Furthermore, the sixth constraint411 f may have a weighting 412 f that indicates that the sixthconstraint 411 f has a lower priority than the first through fourthconstraints 411 a-d, but a higher priority than the fifth constraint 411e. Thus, the sixth constraint 411 f is only followed in certainscenarios, i.e., when resizing of tiles is required to complete the tileinstallation 40. Additionally, instead of adjusting the physical tiles42 based on the fifth constraint 411 e, the control system 218 adjuststhe tile installation process by resizing the physical tiles 42.

The sixth constraint 411 f also provides that, if necessary, only thephysical tiles 42 extending along the bottom edge E_(BOTTOM) or theright edge E_(RIGHT) should be resized. In other words, the designer mayintend to maintain the physical tiles 42 at their initial dimensions h₀,w₀, t₀ as much as possible. Because aspects of the sixth constraint 411f are modifiable, the sixth constraint 411 f also defines a part of theparameter space for the model 400. Here, the modifiable parametersinclude the size of the physical tiles 42 extending along the bottomedge E_(BOTTOM) or the right edge E_(RIGHT). The sizes can be modified,if necessary, to determine adjustments 416 for the tile installationprocess.

Accordingly, as shown in FIG. 5D, the robot 200 mounts the fourthphysical tile 42 d without any resizing. As described above, after thefourth physical tile 42 d is mounted, the control system 218 canre-evaluate and adjust the values, settings, etc., for the entireparameter space defined by the model 400. After mounting the fourthphysical tile 42 d, the robot 200 employs the appropriate sensor(s) 212to measure the spacing between the third physical tile 42 c and thefourth physical tile 42 d. The fourth physical tile 42 d, however, ismounted with another error e₂ and is spaced a distance of D_(C)+e₂ belowthe third physical tile 42 c.

After the robot 200 measures this spacing, the control system 218determines that the fifth physical tile must be resized further toaccommodate the error e₂ in addition to the error e₁. The physical tiles42 a-d combine for a height of 4h₀+3D_(B)+v+e₁+e₂. Thus, the robot 200can employ the appropriate end-effector(s) 211 to resize the fifthphysical tile 42 e to a reduced height h₁ so that the left column C₁fits in the rectangular area A. As shown in FIG. 5E, the fifth physicaltile 42 e is mounted at a distance D_(C) under the fourth physical tile42 d with this reduced height h₁ to complete the left column C₁. Theremaining columns C_(i) and the rows R_(i) can be similarly completedwith the physical tiles 42.

Ideally, the robot 400 produces a tile installation 40 that closelyresembles the model 400 as initially conceived by the designer (shown inFIG. 4A). Aspects of the production process, however, may not beconsistent with some of the assumptions behind the model. For instance,the rectangular area A of the wall 47 above might be slightly smallerthan the model 400 anticipates.

Furthermore, the robot 200 might mount the physical tiles 42 withunanticipated errors. As shown in the example of FIGS. 5A-E, such errors(e.g., errors e₁ and e₂) as well as variations within set tolerances(e.g., variation v) can accumulate during the tile installation processand affect the resulting tile installation 40. Advantageously, the robot200 can monitor each step of the tile installation process. Inparticular, the robot 200 can take various measurements of variousaspects of the tile installation in the three-dimensional space of theinstallation site. By evaluating the measurements at each step, therobot 200 dynamically determines how to mitigate the effect of anyvariations/errors that may be introduced at each step.

In some cases, the robot 200 may decide to abort the tile productionprocess. In other cases, the robot 200 can adjust the tile productionprocess as provided by the specifications 410. The constraints 411 a-fin FIG. 5A reflect the designer's intent while also defining a dynamicdesign space where parameters in the tile installation process can bedynamically modified to accommodate errors that may occur. Inparticular, the robot 200 can resize (e.g., cut) the physical tiles 42to accommodate any errors as each physical tile 42 is mounted. Althoughthe tile installation 40 may differ in some aspects from the model 400shown in FIG. 4A, the tile installation 40 still preserves thedesigner's intent as defined by the constraints 411 a-f.

VI. ADDITIONAL EXAMPLE OF A STRUCTURE PRODUCED BY A ROBOTIC SYSTEM

The example of FIGS. 5A-D is only one example of how the robot 200 canadjust parameters to customize a production process in response tosensor data from the sensors 212. For instance, the robot 200 caninstall heavy construction materials to build a bridge, where the robot200 takes measurements after each installation step and responds tovariations/errors by deciding to abort or modifying the bridgeinstallation process.

FIG. 6 illustrates aspects of another example production process for atable 50. The table 50 is assembled from a table top 52 a and n legs 52b _(i=1, . . . , n) according to specifications 510 generated from amodel 500.

The specifications 510 include constraints 511 and respective weightings512 and tolerances 513. A first constraint 511 a and a respectiveweighing 512 a requires that each leg 52 b, be attached to the lowersurface of the table top 52 a at a respective specifiedposition/orientation S_(i). A second constraint 511 b and a respectiveweighing 512 b requires the upper horizontal surface of the table top 52a to remain level, i.e., generally perpendicular to gravitational forceG. A third constraint 511 c and a respective weighing 512 c requires thetable 50 to support a minimum load (i.e., weight) W acting downwardly atthe center of the table top 52 a. The constraints 511 a-c may alsospecify certain respective tolerances 513.

Using the appropriate end-effector(s) 212, the robot 200 attaches afirst leg 52 b ₁ to the lower surface of the table top 52 a withfasteners, adhesives, welding, mechanical/frictional engagement, and/orother appropriate techniques. Using the appropriate sensor(s) 212, therobot 200 can obtain a measurement M₁ of the position/orientation of thefirst leg 52 b ₁ relative to the table top 52 a.

If the control system 218 detects a variation or error between themeasured position/orientation M₁ and the specified position/orientationS₁, the control system 218 can evaluate the effect of thevariation/error on the production process. In particular, the controlsystem 218 can determine whether each remaining leg 52 b_(i=2, . . . , n) can be attached at its specified position/orientationS_(i=2, . . . , n), while still keeping the table top 52 a levelaccording to the second constraint 511 b and allowing the table 50 tosupport the minimum load W according to the third constraint 511 c. Tomake such a determination, the control system 218 can run a geometricsimulation that predicts the resulting geometry of the table 50 based onattaching the first leg 52 b ₁ at the measured position/orientation M₁and attaching each remaining legs 52 b _(i=2, . . . , n) at itsspecified position/orientation S_(i=2, . . . , n). In addition, thecontrol system 218 can run a mechanical (e.g., statics) simulation todetermine the effect of a load Won the resulting geometry of the table50.

If the control system 218 determines that the second and thirdconstraints 511 b, c can be satisfied with the measuredposition/orientation M₁ of the first leg 52 b ₁, the robot 200 canproceed with the production process by attaching the second leg 52 b ₂according to its specified position/orientation S₂ and so on.

If the control system 218, however, determines that second and thirdconstraints 511 b, c cannot be satisfied with the measuredposition/orientation M₁ of the first leg 52 b ₁, the control system 218can attempt to address the variation or error associated with the firstleg 52 b, by adjusting the position/orientation at which the second leg52 b ₂ and/or subsequent legs 52 b _(i=3, . . . , n) are attached to thetable top 52 a. If possible, the robot 200 proceeds with suchadjustments to the production process to produce the table 50.Otherwise, the robot 200 can abort the production process andcommunicate an alert, so that a user (or other system) can intervene anddetermine how to proceed.

In general, after the attachment of each leg 52 b _(i), the robot 200measures the position/orientation of the attached leg 52 b _(i). Usingthe measurements, the control system 218 can evaluate whether attachingthe remaining legs 52 b _(i+1, . . . , n) according to thespecifications 510 will yield a table top 52 a that is level and a table50 that can support the minimum load W. If necessary, the control system218 can adjust how the remaining legs 52 b _(i+1, . . . , n) should beattached in order to produce the desired table 50. A similar approachcan be employed for other types of physical structures.

VII. CONCLUSION

A robot can use end-effector(s) and sensor(s) to combine a plurality ofphysical objects into a desired physical structure. To produce thisphysical structure, the robot may be guided by specifications that aregenerated from a model for the physical structure. In view of theforegoing, a robot can dynamically adjust aspects of a productionprocess to account for differences between the model and physicalreality. Specifically, the robot can employ sensors to take measurementsafter each step in the production process. Using these measurements, therobot can evaluate whether the production process will produce thedesired physical structure. The specifications derived from the modelprovide the robot with the necessary flexibility to customize theproduction process so that the desired physical structure is produced.The specifications define a flexible design space where differentparameters in the production process can be modified to accommodatephysical reality. Although the adjustments to the production process maychange aspects of the resulting physical structure, the robot operatesso that the production process preserves the intent of the designerprovided in the model.

The above detailed description describes various features and functionsof the disclosed systems, devices, and methods with reference to theaccompanying figures. In the figures, similar symbols typically identifysimilar components, unless context indicates otherwise. The illustrativeimplementations described in the detailed description, figures, andclaims are not meant to be limiting. Other implementations can beutilized, and other changes can be made, without departing from thescope of the subject matter presented herein. It will be readilyunderstood that the aspects of the present disclosure, as generallydescribed herein, and illustrated in the figures, can be arranged,substituted, combined, separated, and designed in a wide variety ofdifferent configurations, all of which are explicitly contemplatedherein.

With respect to any or all of the message flow diagrams, scenarios, andflow charts in the figures and as discussed herein, each step, blockand/or communication may represent a processing of information and/or atransmission of information in accordance with example implementations.Alternative implementations are included within the scope of theseexample implementations. In these alternative implementations, forexample, functions described as steps, blocks, transmissions,communications, requests, responses, and/or messages may be executed outof order from that shown or discussed, including in substantiallyconcurrent or in reverse order, depending on the functionality involved.Further, more or fewer steps, blocks and/or functions may be used withany of the message flow diagrams, scenarios, and flow charts discussedherein, and these message flow diagrams, scenarios, and flow charts maybe combined with one another, in part or in whole.

A step or block that represents a processing of information maycorrespond to circuitry that can be configured to perform the specificlogical functions of a herein-described method or technique.Alternatively or additionally, a step or block that represents aprocessing of information may correspond to a module, a segment, or aportion of program code (including related data). The program code mayinclude one or more instructions executable by a processor forimplementing specific logical functions or actions in the method ortechnique. The program code and/or related data may be stored on anytype of computer-readable medium, such as a storage device, including adisk drive, a hard drive, or other storage media.

A computer-readable medium may include non-transitory computer-readablemedia such as computer-readable media that stores data for short periodsof time like register memory, processor cache, and/or random accessmemory (RAM). The computer-readable media may also includenon-transitory computer-readable media that stores program code and/ordata for longer periods of time, such as secondary or persistent longterm storage, like read only memory (ROM), optical or magnetic disks,and/or compact-disc read only memory (CD-ROM), for example. Thecomputer-readable media may also be any other volatile or non-volatilestorage systems. A computer-readable medium may be considered acomputer-readable storage medium, for example, or a tangible storagedevice.

Moreover, a step or block that represents one or more informationtransmissions may correspond to information transmissions betweensoftware and/or hardware modules in the same physical device. However,other information transmissions may be between software modules and/orhardware modules in different physical devices.

While various aspects and implementations have been disclosed herein,other aspects and implementations will be apparent to those skilled inthe art. The various aspects and implementations disclosed herein arefor purposes of illustration and are not intended to be limiting, withthe true scope being indicated by the following claims.

What is claimed is:
 1. A robotic system, comprising: a body includingone or more end-effectors configured to combine a plurality of objectsin a production process; one or more sensors configured to obtain one ormore measurements relating to the combination of the plurality ofobjects during the production process; and a control system includingone or more processors and one or more data storage devices, the controlsystem communicatively coupled to the one or more sensors, the one ormore data storage devices storing specifications relating to thecombination of the plurality of objects, the control system configuredto: receive, from the one or more sensors, one or more firstmeasurements relating to a combination of a first object and one or moreother objects during the production process; determine a differencebased on the one or more first measurements and the specifications;determine one or more adjustments to the production process based on thedetermined difference; and send, for the one or more end-effectors, oneor more instructions based on the specifications and the one or moreadjustments to the production process, wherein the one or moreend-effectors, in response to the one or more instructions, combines asecond object with the first object and the one or more other objectsbased on the specifications and the one or more adjustments to theproduction process.
 2. The robotic system of claim 1, wherein thespecifications include constraints relating to the combination of theplurality of objects, and the control system is configured to determinethe one or more adjustments to the production process based on theconstraints.
 3. The robotic system of claim 2, wherein thespecifications include respective weightings for the constraints, andthe control system is configured to determine the one or moreadjustments to the production process by selecting, based on theweightings, the one or more adjustments to the production process from aplurality of options.
 4. The robotic system of claim 1, wherein the oneor more end-effectors combines the objects by installing at least someof the objects into or onto one or more external structures.
 5. Therobotic system of claim 1, wherein the one or more first measurementsinclude one or more measurements of a position or an orientation of thefirst object relative to the one or more other objects.
 6. The roboticsystem of claim 5, wherein the one or more adjustments to the productionprocess relate to a position or an orientation of the second objectrelative to the first object and the one or more other objects.
 7. Therobotic system of claim 1, wherein the control system determines the oneor more adjustments to the production process by simulating thecombination of the plurality of objects based on the one or moremeasurements.
 8. The robotic system of claim 1, wherein, in response tothe determined difference, the control system is further configured tosend instructions to cease further action by the one or moreend-effectors, and the one or more end-effectors, in response to the oneor more instructions to cease further action, does not combine thesecond object with the first object and the one or more other objects.9. The robotic system of claim 1, wherein the control system is furtherconfigured to send a report relating to the combination of the firstobject, the second object, and the one or more other structures based onthe specifications and the one or more adjustments to the productionprocess.
 10. The robotic system of claim 1, wherein the one or moresensors are further configured to obtain one or more second measurementsrelating to the combination of the first object, the second object, andthe one or more other structures, the one or more end-effectors isfurther configured to combine a third object with the first object, thesecond object, and the one or more other structures, and the controlsystem is further configured to: receive, from the one or more sensors,the one or more second measurements; determine a second difference basedon the one or more second measurements and the specifications; determineone or more second adjustments to the production process based on thedetermined second difference; and send, for the one or moreend-effectors, one or more second instructions based on thespecifications and the one or more second adjustments to the productionprocess, wherein the one or more end-effectors, in response to the oneor more second instructions, combines the third object with the firstobject, the second object, and the one or more other objects based onthe specifications and the one or more second adjustments to theproduction process.
 11. A method for a robotic system, comprising:combining, with one or more end-effectors, a first object and one ormore other objects during a production process; obtaining, with one ormore sensors, one or more measurements relating to combining the thefirst object and the one or more other objects during the productionprocess; storing, on one or more data storage devices, specificationsfor a combination of a plurality of objects; determining, with a controlsystem including one or more processors, a difference based on the oneor more measurements and the specifications; determining, with thecontrol system, one or more adjustments to the production process basedon the determined difference; and combining, with the one or moreend-effectors, a second object with the first object and the one or moreother objects based on the specifications and the one or moreadjustments to the production process.
 12. The method of claim 11,wherein the specifications include constraints relating to thecombination of the plurality of objects, and the one or more adjustmentsto the production process are determined based on the constraints. 13.The method of claim 11, wherein the one or more measurements include oneor more measurements of a position or an orientation of the first objectrelative to the one or more other objects.
 14. The method of claim 13,wherein the one or more adjustments to the production process relate toa position or an orientation of the second object relative to the firstobject and the one or more other objects.
 15. The method of claim 11,wherein determining the one or more adjustments to the productionprocess includes simulating the combination of the plurality of objectsbased on the one or more measurements.
 16. The method of claim 11,further comprising sending a report relating to the combining the secondobject with the first object and the one or more other objects based onthe specifications and the one or more adjustments to the productionprocess.
 17. The method of claim 11, further comprising: obtaining oneor more second measurements relating to the combination of the firstobject, the second object, and the one or or more other objects,determining, with the control system, a second difference based on thesecond measurements and the specifications; determining, with thecontrol system, one or more second adjustments to the production processbased on the determined second difference; and combining, with the oneor more end-effectors, a third object with the first object, the secondobject, and the one or more other objects based on the specificationsand the one or more second adjustments to the production process.
 18. Amethod for a robotic system, comprising: storing, on one or more datastorage devices, a model for combining a plurality of objects; combininga first set of the objects during a production process; obtaining, withone or more sensors, one or more measurements relating to thecombination of the first set of the objects during the productionprocess; determining, with a control system including one or moreprocessors, one or more differences based on the model and the one ormore measurements; determining, with the control system, one or moreadjustments to the production process based on the one or moredifferences; and combining, with one or more end effectors, a second setof the objects with the first set of objects based on the model and theone or more adjustments to the production process.
 19. The method ofclaim 18, wherein the model includes constraints relating to thecombination of the plurality of objects, and the one or more adjustmentsto the production process are determined based on the constraints. 20.The method of claim 18, wherein determining the one or more adjustmentsincludes simulating the combination of the plurality of objects based onthe one or more measurements.